Ultrasound scanning system

ABSTRACT

A force-controlled ultrasound imaging device uses a linear drive system to maintain a suitable contact force with a subject. A combination of hardware and software limits on travel for the linear drive system provide intuitive user feedback while helping to ensure that the linear drive system remains active throughout the scan, while responding appropriately to initiation and termination of scans.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application No.61/715,406 filed on Oct. 18, 2012, the entire content of which is herebyincorporated by reference.

This application is also related to U.S. patent application Ser. No.12/972,461 filed on Dec. 18, 2010, which claims the benefit of U.S.Provisional Patent Application No. 61/287,886 filed Dec. 18, 2009. Eachof the foregoing applications is hereby incorporated by reference in itsentirety.

FIELD OF THE INVENTION

The invention generally relates to ultrasound imaging.

BACKGROUND

While medical ultrasound images can be quickly and cheaply obtained froma handheld ultrasound imaging device, this type of imaging generallysuffers from a lack of accurate information concerning the conditionsunder which the scan was captured. As a result, two-dimensionalultrasound images from handheld probes are generally limited in use to aqualitative evaluation of the imaged tissue.

There remains a need for an ultrasound imaging device that captures anacquisition state for a scan including quantitative data about theconditions under which the scan was obtained such as a position of ascanner or a contact force applied when an image is obtained.

SUMMARY

A force-controlled ultrasound imaging device uses a linear drive systemto maintain a suitable contact force with a subject. A combination ofhardware and software limits on travel for the linear drive systemprovide intuitive user feedback while helping to ensure that the lineardrive system remains active throughout the scan, while respondingappropriately to initiation and termination of scans.

BRIEF DESCRIPTION OF THE FIGURES

The invention and the following detailed description of certainembodiments thereof may be understood by reference to the followingfigures, wherein similar reference characters denote similar elementsthroughout the several views.

FIG. 1 is a perspective view of a handheld ultrasound probe controldevice.

FIG. 2 is a schematic view of a handheld ultrasound probe.

FIG. 3 is a flowchart of a process for force-controlled acquisition ofultrasound images.

FIG. 4 shows a lumped parameter model of the mechanical system of aprobe as described herein.

FIG. 5 is a flowchart depicting operating modes of a force-controlledultrasound probe.

FIG. 6 shows a process for ultrasound image processing.

FIG. 7 is a schematic view of an ultrasound scanning system.

FIG. 8 is a flowchart for a process for obtaining a reconstructed volumeof a target using a handheld ultrasound probe.

FIG. 9 is a flowchart for a process for capturing an acquisition statefor an ultrasound scan.

FIG. 10 shows a fiducial marker according to an embodiment.

FIG. 11 shows a generalized workflow using acquisition states accordingto an embodiment.

FIG. 12 depicts hard limits and soft limits for controlling the travelof a probe.

FIG. 13 shows several operating modes for a force-controlled probe.

FIG. 14 is a block diagram of a force and position control module.

FIG. 15 depicts a system to provide an indication of probe positionrelative to the end of travel.

FIG. 16 shows a force-controlled probe.

DETAILED DESCRIPTION

The embodiments will now be described more fully hereinafter withreference to the accompanying figures, in which preferred embodimentsare shown. This invention may, however, be embodied in many differentforms and should not be construed as limited to the illustratedembodiments set forth herein. Rather, these illustrated embodiments areprovided so that this disclosure will convey the scope of the inventionto those skilled in the art.

All documents mentioned herein are hereby incorporated in their entiretyby reference. References to items in the singular should be understoodto include items in the plural, and vice versa, unless explicitly statedotherwise or clear from the text. Grammatical conjunctions are intendedto express any and all disjunctive and conjunctive combinations ofconjoined clauses, sentences, words, and the like, unless otherwisestated or clear from the context. Thus, the term “or” should generallybe understood to mean “and/or” and so forth.

Recitation of ranges of values herein are not intended to be limiting,referring instead individually to any and all values falling within therange, unless otherwise indicated herein, and each separate value withinsuch a range is incorporated into the specification as if it wereindividually recited herein. The word “about,” when accompanying anumerical value, is to be construed as indicating a deviation as wouldbe appreciated by one of ordinary skill in the art to operatesatisfactorily for an intended purpose. Ranges of values and/or numericvalues are provided herein as examples only, and do not constitute alimitation on the scope of the described embodiments. The use of any andall examples, or exemplary language (“e.g.,” “such as,” or the like)provided herein, is intended merely to better illuminate the embodimentsand does not pose a limitation on the scope of the embodiments. Nolanguage in the specification should be construed as indicating anyunclaimed element as essential to the practice of the embodiments.

In the following description, like reference characters designate likeor corresponding parts throughout the figures. Additionally, in thefollowing description, it is understood that terms such as “front,”“back,” “first,” “second” and the like, are words of convenience and arenot to be construed as limiting terms.

The techniques described herein enable real-time control of the contactforce between an ultrasound probe and a target, such as a patient'sbody. Specifically, techniques described herein enable real-time controlthrough the use of a force sensor, a position sensor, and a linear drivesystem that moves the ultrasound probe in response to a control signalprovided by a controller module using information from the force andposition sensors. This allows ultrasound technicians to take fixed- orvariably-controlled-contact-force ultrasound measurements of the target,as desired. This also facilitates measurement, tracking, and/or controlof the contact force in a manner that permits enhanced, quantitativeanalysis and subsequent processing of ultrasound image data.

FIG. 1 is a perspective view of a handheld ultrasound probe controldevice. The device 100 may include a frame 118 adapted to receive aprobe 112, a linear drive system 122 that translates the frame 118 alongan actuation axis 114, a sensor 110 such as a force sensor, a torquesensor, position sensor 142 or some combination of these, and acontroller 120. The device may also include a linear drive system thatpositions the frame 118 in other directions.

The probe 112 can be of any known type or construction. The probe 112may, for example, include a handheld ultrasound probe used for medicalimaging or the like. More generally, the probe 112 may include anycontact scanner or other device that can be employed in a manner thatbenefits from the systems and methods described herein. Thus, oneadvantage of the device 100 is that a standard off-the-shelf ultrasoundmedical probe can be retrofitted for use as a force-controlledultrasound in a relatively inexpensive way, i.e., by mounting the probe112 in the frame 118. Medical ultrasound devices come in a variety ofshapes and sizes, and the frame 118 and other components may be adaptedfor a particular size/shape of the probe 112, or may be adapted toaccommodate varying sizes and/or shapes. In another aspect, the probe112 may be integrated into the frame 118 or otherwise permanentlyaffixed to (or in) the frame 118.

In general, a probe 112, such as an ultrasound probe, includes anultrasound transducer 124. The construction of suitable ultrasoundtransducers is generally well known. In one aspect, an ultrasoundtransducer includes piezoelectric crystals or similar means to generateultrasound waves and/or detect incident ultrasound. More generally, anysuitable arrangement for transmitting and/or receiving ultrasound may beused as the ultrasound transducer 124 in the embodiments describedherein. Still more generally, other transceiving mechanisms ortransducers may also or instead be used to support imaging modalitiesother than ultrasound.

The frame 118 may include any substantially rigid structure thatreceives and holds the probe 112 in a fixed position and orientationrelative to the frame 118. The frame 118 may include an opening thatallows an ultrasound transducer 124 of the probe 112 to contact apatient's skin or other surface through which ultrasound images are tobe obtained. Although FIG. 1 shows the probe 112 held within the frame118 between two plates (a front plate 128 and a larger plate 130, wherethe front plate 128 is bolted to the larger plate 130 on the frame 118)arranged to surround a handheld ultrasound probe and securely affix theprobe to the frame 118, any suitable technique may also or instead beemployed to secure the probe 112 in a fixed relationship to the frame118. For example, the probe 112 may be secured with a press fit, hooks,screws, anchors, adhesives, magnets, or the like, or any combination ofthese and other fasteners. More generally, the frame 118 may include anystructure or combination of structures suitable for securely retainingthe probe 112 in a fixed positional relationship relative to the probe112, for example, including but not limited to a sleeve, pocket, or thelike.

In one aspect, the frame 118 may be adapted for handheld use, and moreparticularly adapted for gripping by a technician in the sameorientation as a conventional ultrasound probe. Without limitation, thismay include a trunk 140 or the like for gripping by a user that extendsaxially away from the ultrasound transducer 124 and generally normal tothe contact surface of the transducer 124. Stated alternatively, thetrunk 140 may extend substantially parallel to the actuation axis 114and be shaped and sized for gripping by a human hand. In this manner,the trunk 140 may be gripped by a user in the same manner andorientation as a typical handheld ultrasound probe. The linear drivesystem 122 may advantageously be axially aligned with the trunk 140 topermit a more compact design consistent with handheld use. That is, aballscrew or similar linear actuator may be aligned to pass through thetrunk 140 without diminishing or otherwise adversely affecting the rangeof linear actuation.

The linear drive system 122 may be mounted on the device 100 and mayinclude a control input electronically coupled to the controller 120.The linear drive system 122 may be configured to translate the probe 112along an actuation axis 114 in response to a control signal from thecontroller 120 to the control input of the linear drive system 122.Additionally, the linear drive system 122 may include a position sensor142 to provide position information characterizing a position of theprobe 112 along the actuation axis 114. The position may, for example,be a position relative to one or more travel limits of the linear drivesystem 122. Although the linear drive system 122 is depicted by way ofexample as a motor 102 and a linear actuator 104, any system capable oflinearly moving the probe 112 can be employed. For example, the lineardrive system 122 can include a mechanical actuator, hydraulic actuator,pneumatic actuator, piezoelectric actuator, electro-mechanical actuator,linear motor, telescoping linear actuator, ballscrew-driven linearactuator, or the like. More generally, any actuator or combination ofactuators suitable for use within a grippable handheld form factor suchas the trunk 140 may be suitably employed as the linear drive system122. Moreover, the linear drive system 122 may include gears, motors,rails, or the like, in addition to, or in lieu of, linear actuators. Insome implementations, the linear drive system 122 is configured to havea low backlash (e.g., less than 3 μm) or no backlash in order to improvepositional accuracy and repeatability.

The ability of the probe 112 to travel along the actuation axis 114permits the technician some flexibility in hand placement while usingthe device 100. In some implementations, the probe 112 can travel up tosix centimeters along the actuation axis 114, although greater or lesserranges of travel may be readily accommodated with suitable modificationsto the linear actuator 104 and other components of the device 100.

The motor 102 may be electrically coupled to the controller 120 andmechanically coupled in a fixed positional relationship to the linearactuator 104. The motor 102 may be configured to drive the linearactuator 104 in response to control signals from the controller 120, asdescribed more fully herein. The motor 102 can include a servo motor, aDC stepper motor, a hydraulic pump, a pneumatic pump, or the like.

The sensor 110, which may include a force sensor and/or a torque sensor,may be mechanically coupled to the frame 118, such as in a fixedpositional relationship to sense forces/torques applied to the frame118. The sensor 110 may also be electronically coupled to the controller120, and configured to sense a contact force between the probe 112 and atarget surface (also referred to herein simply as a “target”) such as abody from which ultrasound images are to be captured. As depicted, thesensor 110 may be positioned between the probe 112 and the back plate ofthe frame 118. Other deployments of the sensor 110 are possible as longas the sensor 110 is capable of detecting the contact force (for a forcesensor) between the probe 112 and the target surface. Embodiments of thesensor 110 may also or instead include a multi-axis force/torque sensor,a plurality of separate force and/or torque sensors, or the like.

The force sensor may be mechanically coupled to the ultrasound probe 112and configured to obtain a pressure applied by the ultrasound probe 112to the skin surface or a target 136. The force sensor may include apressure transducer coupled to the ultrasound probe 112 and configuredto sense an instantaneous contact force between the handheld ultrasoundprobe 112 and the skin.

The sensor 110 can provide output in any known form, and generallyprovides a signal indicative of forces and/or torques applied to thesensor 110. For example, the sensor 110 can produce analog output suchas a voltage or current proportional to the force or torque detected.Alternatively, the sensor 110 may produce digital output indicative ofthe force or torque detected. Moreover, digital-to-analog oranalog-to-digital converters (not shown) can be deployed at any pointbetween the sensors and other components to convert between these modes.Similarly, the sensor 110 may provide radio signals (e.g., for wirelessconfigurations), optical signals, or any other suitable output that cancharacterize forces and/or torques for use in the device 100 describedherein.

The controller 120 may generally include processing circuitry to controloperation of the device 100 as described herein. The controller 120 mayreceive signals from the sensor 110 indicative of force/torque and fromthe position sensor 142 of the linear drive system 122 indicative of theposition of the probe 112 relative to the travel end points, and maygenerate a control signal to a control input of the linear drive system122 (or directly to the linear actuator 104) for maintaining a givencontact force between the ultrasound probe 112 and the target, asdescribed more fully herein. The controller 120 may include analog ordigital circuitry, computer program code stored in a non-transitorycomputer-readable storage medium, and the like. Embodiments of thecontroller 120 may employ pure force control, impedance control, contactforce-determined position control, and the like.

The controller 120 may be configured with preset limits relating tooperational parameters such as force, torque, velocity, acceleration,position, current, and the like, so as to immediately cut power from thelinear drive system 122 when any of these operational parameters exceedthe preset limits. In some implementations, these preset limits aredetermined based on the fragility of the target. For example, one set ofpreset limits may be selected where the target is a healthy humanabdomen, another set of preset limits may be selected where the targetis a human abdomen of an appendicitis patient, etc. In addition, presetlimits for operational parameters may be adjusted to accommodatediscontinuities such as initial surface contact or termination of anultrasound scan (by breaking contact with a target surface).

In some implementations, the device 100 includes a servo-motor-drivenballscrew linear actuator comprising a MAXON servo motor (EC-Max#272768) (motor 102) driving an NSK MONOCARRIER compact ballscrewactuator (linear actuator 104). a MINI40 six-axis force/torque sensor(sensor 110) from ATI INDUSTRIAL AUTOMATION, which simultaneouslymonitors all three force and all three torque axes, may be mounted tothe carriage of the actuator, and a TERASON 5 MHz ultrasound transducer(ultrasound transducer 124) may be mounted to the force/torque sensor.

The vector from a geometric origin of the sensor 110 to an endpoint atthe probe 124 that contacts a patient can be used to map the forces andtorques at the sensor 110 into the contact forces and torques seen atthe probe/patient interface. In some implementations, it is possible tomaintain a set contact force with a mean error of less than about 0.2%and, in a closed-loop system, maintain a desired contact force with amean steady state error of about 2.1%, and attain at least about 20Newtons of contact force. More generally, in an embodiment a steadystate error of less than 3% was achieved for applied forces ranging fromone to seven Newtons.

Other sensors (indicated generically as a second sensor 138) may beincluded without departing from the scope of this invention. Forexample, a second sensor 138 such as an orientation sensor or the likemay be included, which may be operable to independently detect at leastone of a position and an orientation of the device 100, such as to tracklocation and/or orientation of the device 100 before, during, and afteruse. This data may help to further characterize operation of the device100. A second sensor 138 such as a range or proximity detector may beemployed to anticipate an approaching contact surface and place thedevice 100 in a state to begin an ultrasound scan. For example, aproximity sensor may be operable to detect a proximity of the ultrasoundtransducer 124 to a subject (e.g., the target surface). One or moreinertial sensors may be included in the device 100. Suitable inertialsensors include, for example, inertial sensors based on MEMS technologysuch as accelerometers and gyroscopes, or any other device orcombination of devices that measure motion. More generally, any of avariety of sensors known in the art may be used to augment or supplementoperation of the device 100 as contemplated herein.

The ultrasound probe may further include a sensor for illuminating theskin surface when the handheld ultrasound probe is placed for useagainst the skin surface. For example, the sensor may be a lightingsource mechanically coupled to the handheld ultrasound probe andpositioned to illuminate the skin surface during use of the ultrasoundprobe. The lighting source may be part of the sensor system of theultrasound probe or the lighting source may be a separate devicedirected toward the ultrasound probe. Suitable lighting sources includean LED light or any other light capable of illuminating the skin surfaceduring ultrasound scanning.

Another sensor that may be included in the device 100 is a camera 132.The camera 132 may be positioned to record a digital image of the target136 during an ultrasound scan when the handheld ultrasound probe 112 isplaced for use against the skin surface 137 of the target 136. Thecamera 132 also may be positioned to obtain a pose of the handheldultrasound probe 112 as the ultrasound transducer 124 scans the target136. The camera 132 may be mechanically coupled to the ultrasoundtransducer 124. In one aspect, the camera 132 may be rigidly mounted tothe ultrasound transducer 124 and directed toward the skin surface 137(when positioned for use) in order to capture images of the skin surface137 and/or a target 134 adhered to the skin surface 137. In anotheraspect, the camera 132 may be mounted separate from the ultrasound probe112 and directed toward an area of use of the ultrasound probe 112 sothat the camera 132 can capture images of the ultrasound probe 112 inorder to derive pose information directly from images of the ultrasoundprobe 112. Suitable cameras 132 may, for example, include anycommercially available digital camera or digital video camera designedto capture images of sufficient quality for use as contemplated herein.The lighting source, sensor(s), and/or camera(s) may also include imageintensification, magnification, active illumination, thermal imaging, orthe like

The ultrasound probe 112 may have an integral structure with variouscomponents coupled directly to a body thereof, or one or more of thevarious functions of one or more of the components of the ultrasoundprobe may be distributed among one or more independent devices. Forexample, the camera, the lighting source, and any other sensors may beintegrated into the ultrasound probe or they may be separate from theultrasound probe, along with suitable communications and control systemswhere coordination of function is desired between the probe and theexternal components.

The ultrasound probe 112 may be used to capture an ultrasound image of atarget 136 through a skin surface 137. A fiducial marker 134 withpredetermined dimensions may be applied to the skin surface 137 of thetarget 136 that is to be scanned by the ultrasound probe 112. Thefiducial marker 134 may have any desired dimension or shape such as asquare, a rectangle, a circle and/or any other regular, irregular,and/or random shape and/or patterns. In an embodiment, the fiducialmarker 134 may be a 3 mm×3 mm square. The fiducial marker 134 may bemade of a thin material. Suitable materials include, but are not limitedto, any materials that will not obstruct the transducer from obtainingan ultrasound scan of the target 136. The fiducial marker 134 may beadhered to the skin surface 137 of a target 136 using any suitablemethods and/or any suitable adhesives. In another aspect, the fiducialmarker 134 may be stamped, inked or otherwise applied to the skinsurface using ink or any other suitable visually identifiable markingmaterial(s).

FIG. 2 is a schematic depiction of an ultrasound probe. The probe 200,which may be a force-controlled ultrasound probe, generally includes asensor 110, a controller 120, a linear drive system 122, a positionsensor 142, and an ultrasound transducer 124 as described herein.

In contrast to the probe 112 mounted in the device 100 as described inFIG. 1, the probe 200 of FIG. 2 may have the sensor 110, controller 120,and linear drive system 122 integrally mounted (as opposed to mounted ina separate device 100) in a single device to provide a probe 200 with anintegral structure. In FIG. 2, the components are all operable to gatherultrasound images at measured and/or controlled forces and torques, asdescribed with reference to FIG. 1. More generally, the variousfunctions of the above-described components may be distributed acrossseveral independent devices in various ways (e.g., an ultrasound probewith integrated force/torque sensors but an external drive system, anultrasound probe with an internal drive system but with an externalcontrol system, etc.). In one aspect, a wireless handheld probe 200 maybe provided that transmits sensor data and/or ultrasound data wirelesslyto a remote computer that captures data for subsequent analysis anddisplay. All such permutations of the features described herein arewithin the scope of this disclosure.

The ultrasound transducer 124 can include a medical ultrasonictransducer, an industrial ultrasonic transducer, or the like. Like theultrasound probe 112 described with reference to FIG. 1, it will beappreciated that a variety of embodiments of the ultrasound transducer124 are possible, including embodiments directed to non-medicalapplications such as nondestructive ultrasonic testing of materials andobjects and the like, or more generally, transducers or othertransceivers or sensors for capturing data instead of or in addition toultrasound data. Thus, although reference is made to an “ultrasoundprobe” in this document, the techniques described herein are moregenerally applicable to any context in which the transmission of energy(e.g., sonic energy, electromagnetic energy, thermal energy, etc.) fromor through a target varies as a function of the contact force betweenthe energy transmitter and the target.

Other inputs/sensors may be usefully included in the probe 200. Forexample, the probe 200 may include a limit switch 202 or multiple limitswitches 202. These may be positioned at any suitable location(s) todetect limits of travel of the linear drive system 122, and may be usedto prevent damage or other malfunction of the linear drive system 122 orother system components. The limit switch(es) may be electronicallycoupled to the controller 120 and provide a signal to the controller 120to indicate when the limit switch 202 detects an end of travel of thelinear drive system along the actuation axis. The limit switch 202 mayinclude any suitable electro-mechanical sensor or combination of sensorssuch as a contact switch, proximity sensor, range sensor, magneticcoupling, and the like.

The position sensor 142 may be electronically and/or mechanicallycoupled to the limit switch(es) 202 to provide positioning informationto the controller 120 concerning physical limits marked by the limitswitch(es) 202. The position sensor 142 may provide position informationto the controller 120 by tracking the travel of the probe 200 along theactuation axis 114 relative to the limit switch(es) 202. The controller120 may receive the position information from the position sensor 142,determine the position of the probe 200, e.g., relative to the limitswitch positions, and may provide control signals to movement of theprobe 200 too close to the limit(s) of travel defined by the limitswitch(es) 202.

The probe 200 may also or instead include one or more user inputs 204.These may be physically realized by buttons, switches, dials, or thelike on the probe 200. The user inputs 204 may be usefully positioned invarious locations on an exterior of the probe 200. For example, the userinputs 204 may be positioned where they are readily finger-accessiblewhile gripping the probe 200 for a scan. In another aspect, the userinputs 204 may be positioned away from usual finger locations so thatthey are not accidentally activated while manipulating the probe 200during a scan. The user inputs 204 may generally be electronicallycoupled to the controller 120, and may support or activate functionssuch as initiation of a scan, termination of a scan, selection of acurrent contact force as the target contact force, storage of a currentcontact force in memory for subsequent recall, or recall of apredetermined contact force from memory. Thus, a variety of functionsmay be usefully controlled by a user with the user inputs 204.

A memory 210 may be provided to store ultrasound data from theultrasound transducer and/or sensor data acquired from any of thesensors during an ultrasound scan. The memory 210 may be integrallybuilt into the probe 200 to operate as a standalone device, or thememory 210 may include remote storage, such as in a desktop computer,network-attached storage, or other device with suitable storagecapacity. In one aspect, data may be wirelessly transmitted from theprobe 200 to the memory 210 to permit wireless operation of the probe200. The probe 200 may include any suitable wireless interface 220 toaccommodate such wireless operation, such as for wireless communicationswith a remote storage device (which may include the memory 210). Theprobe 200 may also or instead include a wired communications interfacefor serial, parallel, or networked communication with externalcomponents. The memory 210 may be used to return the device to apreviously recorded acquisition state.

A display 230 may be provided, which may receive wired or wireless datafrom the probe 200. The display 230 and memory 210 may be a display andmemory of a desktop computer or the like, or may be standaloneaccessories to the probe 200, or may be integrated into a medicalimaging device that includes the probe 200, memory 210, display 230, andany other suitable hardware, processor(s), and the like. The display 230may display ultrasound images obtained from the probe 200 using knowntechniques. The display 230 may also or instead display a currentcontact force or instantaneous contact force measured by the sensor 110,which may be superimposed on a corresponding ultrasound image or inanother display region of the display 230. Other useful information,such as a target contact force, an actuator displacement, or anoperating mode, may also or instead be usefully rendered on the display230 to assist a user in obtaining ultrasound images.

A processor 250 may also be provided. In one aspect, the processor 250,memory 210, and display 230 are a desktop or laptop computer. In anotheraspect, these components may be separate, or there may exist somecombination of these. For example, the display 230 may be a supplementaldisplay provided for use by a doctor or technician during an ultrasoundscan. The memory 210 may be a network-attached storage device or thelike that logs ultrasound images and other acquisition state data. Theprocessor 250 may be a local or remote computer provided for post-scanor in-scan processing of data. In general, the processor 250 and/or arelated computing device may have sufficient processing capability toperform the quantitative processing described herein. For example, theprocessor 250 may be configured to process an image of a subject fromthe ultrasound transducer 124 of the probe 200 to provide an estimatedimage of the subject at a predetermined contact force of the ultrasoundtransducer. This may, for example, be an estimate of the image at zeroNewtons (no applied force), or an estimate of the image at some positivevalue (e.g., one Newton) selected to normalize a plurality of imagesfrom the ultrasound transducer 124. Details of this image processing areprovided herein by way of example with reference to FIG. 6.

FIG. 3 is a flowchart of a process for force-controlled acquisition ofultrasound images. The process 300 can be performed, e.g., using ahandheld ultrasound probe 112 mounted in a device 100, or a handheldultrasound probe 200 with integrated force control hardware, or thelike.

As shown in step 302, the process 300 may begin by calibrating the forceand/or torque sensors. The calibration step is for minimizing (orideally, eliminating) errors associated with the weight of theultrasound probe or the angle at which the sensors are mounted withrespect to the ultrasound transducer, and may be performed using avariety of calibration techniques known in the art.

To compensate for the mounting angle, the angle between the sensor axisand the actuation axis may be independently measured (e.g., when thesensor is installed). This angle may be subsequently stored for use bythe controller to combine the measured forces and/or torques along eachaxis into a single vector, using standard coordinate geometry. Forexample, for a mounting angle θ, scaling the appropriate measured forcesby sin(θ) and cos(θ) prior to combining them.

To compensate for the weight of the ultrasound probe, a baselinemeasurement may be taken, during a time at which the ultrasound probe isnot in contact with the target. Any measured force may be modeled as dueeither to the weight of the ultrasound probe, or bias inherent in thesensors. In either case, the baseline measured force may be recorded,and may be subtracted from any subsequent force measurements. Where dataconcerning orientation of the probe is available, this compensation mayalso be scaled according to how much the weight is contributing to acontact force normal to the contact surface. Thus, for example, an imagefrom a side (with the probe horizontal) may have no contribution tocontact force from the weight of the probe, while an image from a top(with the probe vertical) may have the entire weight of the probecontributing to a normal contact force. This variable contribution maybe estimated and used to adjust instantaneous contact force measurementsobtained from the probe.

As shown in step 304, a predetermined desired force may be identified.In some implementations, the desired force is simply a constant force.For example, in imaging a human patient, a constant force of less thanor equal 20 Newtons is often desirable for the comfort and safety of thepatient.

In some implementations, the desired force may vary as a function oftime. For example, it is often useful to “poke” a target in a controlledmanner, and acquire images of the target as it deforms during or afterthe poke. The desired force may also or instead include a desired limit(minimum or maximum) to manually applied force. In some implementations,the desired force may involve a gradual increase of force given by afunction F(t) to a force F_(max) at a time G_(max), and then a symmetricreduction of force until the force reaches zero. Such a function isoften referred to as a “generalized tent map,” and may be given by thefunction G(t)=F(t) if t<t_(max), and G(t)=F_(max)−F(t−t_(max)) fort≧t_(max). When F is a linear function, the graph of G(t) resembles atent, hence the name. In another aspect, a desired force function mayinvolve increasing the applied force by some function F(t) for aspecified time period until satisfactory imaging (or patient comfort) isachieved, and maintaining that force thereafter until completion of ascan. The above functions are given by way of example. In general, anypredetermined force function can be used.

As shown in step 306, the output from the force and/or torque andposition sensors may be read as sensor inputs to a controller or thelike.

As shown in step 308, these sensor inputs may be compared to the desiredforce function to determine a force differential. In someimplementations, the comparison can be accomplished by computing anabsolute measure such as the difference of the sensor output with thecorresponding desired sensor output. Similarly, a relative measure suchas a ratio of output to the desired output can be computed.Additionally, output from the position sensor 142 may be compared to thepositions of the limit switch(es) 202 to determine if the probe 200 isapproaching an end of travel of the linear drive system 122. Further,many other functions can be used.

As shown in step 310, a control signal may be generated based on thecomparison of actual-to-desired sensor outputs (or, from the perspectiveof a controller/processor, sensor inputs). The control signal may besuch that the linear drive system is activated in such a way as to causethe measured force and/or torque to be brought closer to a desired forceand/or torque at a given time. For example, if a difference between themeasured force and the desired force is computed, then the drive systemcan translate the probe with a force whose magnitude is proportional tothe difference, and in a direction to reduce or minimize the difference.Similarly, if a ratio of the desired force and measured force iscomputed, then the drive system can translate the probe with a forcewhose magnitude is proportional to one minus this ratio.

More generally, any known techniques from control theory can be used todrive the measured force towards the desired force. These techniquesinclude linear control algorithms, proportional-integral-derivative(“PID”) control algorithms, fuzzy logic control algorithms, etc. By wayof example, the control signal may be damped in a manner that avoidssharp movements of the probe against a patient's body. In anotheraspect, a closed-loop control system may be adapted to accommodateordinary variations in a user's hand position. For example, a human handtypically has small positional variations with an oscillating frequencyof about four Hertz to about twenty Hertz. As such, the controller maybe configured to compensate for an oscillating hand movement of a userat a frequency between four Hertz and thirty Hertz or any other suitablerange. Thus, the system may usefully provide a time resolution finerthan twenty Hertz or thirty Hertz, accompanied by an actuation rangewithin the time resolution larger than typical positional variationsassociated with jitter or tremors in an operator's hand.

As shown in step 312, the ultrasound probe can acquire an image, afraction of an image, or more than one image. It will be understood thatthis may generally occur in parallel with the force control stepsdescribed herein, and images may be captured at any suitable incrementindependent of the time step or time resolution used to provide forcecontrol. The image(s) (or fractions thereof) may be stored together withcontact force and/or torque information (e.g., instantaneous contactforce and torque) applicable during the image acquisition. In someimplementations, the contact force and/or torque information includesall the information produced by the force and/or torque sensors, such asthe moment-by-moment output of the sensors over the time period duringwhich the image was acquired. In some implementations, other derivedquantities can be computed and stored, such as the average or meancontact force and/or torque, the maximum or minimum contact force and/ortorque, and the like.

It will be understood that the steps of the methods described herein maybe varied in sequence, repeated, modified, or deleted, or additionalsteps may be added, all without departing from the scope of thisdisclosure. By way of example, the step of identifying a desired forcemay be performed a single time where a constant force is required, orcontinuously where a time-varying applied force is desired. Similarly,measuring contact force may include measuring instantaneous contactforce or averaging a contact force over a sequence of measurementsduring which an ultrasound image is captured. In addition, operation ofthe probe in clinical settings may include various modes of operationeach having different control constraints. Some of these modes aredescribed herein with reference to FIG. 5. Thus, the details of theforegoing will be understood as non-limiting examples of the systems andmethods of this disclosure.

FIG. 4 shows a lumped parameter model of the mechanical system of aprobe as described herein. While a detailed mathematical derivation isnot provided, and the lumped model necessarily abstracts away somecharacteristics of an ultrasound probe, the model of FIG. 4 provides auseful analytical framework for creating a control system that can berealized using the controller and other components described herein toachieve force-controlled acquisition of ultrasound images.

In general, the model 400 characterizes a number of lumped parameters ofa controlled-force probe. The physical parameters for an exemplaryembodiment are as follows. M_(u) is the mass of ultrasound probe andmounting hardware, which may be about 147 grams. M_(c) is the mass of aframe that secures the probe, which may be about 150 grams. M_(s) is themass of the linear drive system, which may be about 335 grams. k_(F/T)is the linear stiffness of a force sensor, which may be about 1.1*10⁵N/m. k_(e) is the target skin stiffness, which may be about 845N/m.b_(e) is the viscous damping coefficient of the target, which may beabout 1500 Ns/m. k_(t) is the user's total limb stiffness, which may beabout 1000 N/m. b_(t) is the user's total limb viscous dampingcoefficient, which may be about 5000 Ns/m. b_(c) is the frame viscousdamping coefficient, which may be about 0 Ns/m. k_(C) is the stiffnessof the linear drive system, which may be about 3*10⁷ for an exemplaryballscrew and nut drive. K_(T) is the motor torque constant, which maybe about 0.0243 Nm/A. β_(b) is be the linear drive system viscousdamping, which may be about 2*10⁻⁴ for an exemplary ballscrew and motorrotor. L is the linear drive system lead, which may be about 3*10⁻⁴ foran exemplary ballscrew. J_(tot) is the moment of inertia, which may beabout 1.24*10⁻⁶ kgm² for an exemplary ballscrew and motor rotor.

Using these values, the mechanical system can be mathematically modeled,and a suitable control relationship for implementation on the controllercan be determined that permits application of a controlled force to thetarget surface by the probe. Stated differently, the model may beemployed to relate displacement of the linear drive system to appliedforce in a manner that permits control of the linear drive system toachieve an application of a controlled force to the target surface. Itwill be readily appreciated that the lumped model described herein isprovided by way of illustration and not limitation. Variations may bemade to the lumped model and the individual parameters of the model,either for the probe described herein or for probes having differentconfigurations and characteristics, and any such model may be usefullyemployed provided it yields a control model suitable for implementationon a controller as described herein.

FIG. 5 is a flowchart depicting operating modes of a force-controlledultrasound probe. While the probe described herein may be usefullyoperated in a controlled-force mode as discussed herein, use of thehandheld probe in clinical settings may benefit from a variety ofadditional operating modes for varying circumstances such as initialcontact with a target surface or termination of a scan. Several usefulmodes are now described in greater detail.

In general, the process 500 includes an initialization mode 510, a scaninitiation mode 520, a controlled-force mode 530, and a scan terminationmode 540, ending in termination 550 of the process 500.

As shown in step 510, an initialization may be performed on a probe.This may include, for example, powering on various components of theprobe, establishing a connection with remote components such as adisplay, a memory, and the like, performing any suitable diagnosticchecks on components of the probe, and moving a linear drive system to aneutral or ready position, which may for example be at a mid-point of arange of movement along an actuation axis.

As shown in step 522, the scan initiation mode 520 may begin bydetecting a force against the probe using a sensor, such as any of thesensors described herein. In general, prior to contact with a targetsurface such as a patient, the sensed force may be at or near zero. Inthis state, it would be undesirable for the linear drive system to moveto a limit of actuation in an effort to achieve a target controlledforce. As such, the linear drive system may remain inactive and in aneutral or ready position during this step.

As shown in step 524, the controller may check to determine whether theforce detected in step 522 is at or near a predetermined contact forcesuch as the target contact force for a scan. If the detected force isnot yet at (or sufficiently close to) the target contact force, then theinitiation mode 520 may return to step 522 where an additional forcemeasurement is acquired. If the force detected in step 522 is at or nearthe predetermined contact force, the process 500 may proceed to thecontrolled-force mode 530. Thus, a controller disclosed herein mayprovide an initiation mode in which a linear drive system is placed in aneutral position and a force sensor is measured to monitor aninstantaneous contact force, the controller transitioning tocontrolled-force operation when the instantaneous contact force meets apredetermined threshold. The predetermined threshold may be thepredetermined contact force that serves as the target contact force forcontrolled-force operation, or the predetermined threshold may be someother limit such as a value sufficiently close to the target contactforce so that the target contact force can likely be readily achievedthrough actuation of the linear drive system. The predeterminedthreshold may also or instead be predictively determined, such as bymeasuring a change in the measured contact force and extrapolating(linearly or otherwise) to estimate when the instantaneous contact forcewill equal the target contact force.

As shown in step 532, the controlled-force mode 530 may begin byinitiating controlled-force operation, during which a control system maybe executed in the controller to maintain a desired contact forcebetween the probe and a target, all as generally discussed herein.

While in the controlled-force mode 530, other operations may beperiodically performed. For example, as shown in step 534, the currentcontact force may be monitored for rapid changes. In general, a rapiddecrease in contact force may be used to infer that a probe operator hasterminated a scan by withdrawing the probe from contact with a targetsurface. This may be, for example, a step decrease in measured force tozero, or any other pattern of measured force that deviates significantlyfrom expected values during an ongoing ultrasound scan. If there is arapid change in force, then the process 500 may proceed to thetermination mode 540. It will be appreciated that this transition may beterminated where the force quickly returns to expected values, and theprocess may continue in the controlled-force mode 530 even where thereare substantial momentary variations in measured force. As shown in step536, limit detectors for a linear drive system may be periodically (orcontinuously) monitored to determine whether an actuation limit of thelinear drive system has been reached. If no such limit has been reached,the process 500 may continue in the controlled-force mode 530 byproceeding for example to step 537. In one example, if an actuationlimit has been reached, then the process may proceed to termination 550where the linear drive system is disabled. In another example, if anactuation limit has been reached, an endpoint avoidance strategy may beenabled to maintain the current position and force as described in moredetail in FIG. 12. It will be appreciated that the process 500 mayinstead proceed to the termination mode 540 to return the linear drivesystem to a neutral position for future scanning.

As shown in step 537, a contact force, such as a force measured with anyof the force sensors described herein, may be displayed on a monitor orthe like. It will be appreciated that the contact force may be aninstantaneous contact force or an average contact force for a series ofmeasurements over any suitable time interval. The contact force may, forexample, be displayed alongside a target contact force or other data. Asshown in step 538, ultrasound images may be displayed using any knowntechnique, which display may be alongside or superimposed with the forcedata and other data described herein.

As shown in step 542, when a rapid force change or other implicit orexplicit scan termination signal is received, the process 500 may entera scan termination mode 540 in which the linear drive system returns toa neutral or ready position using any suitable control algorithm, suchas a controlled-velocity algorithm that returns to a neutral position(such as a mid-point of an actuation range) at a constant, predeterminedvelocity. When the linear drive system has returned to the readyposition, the process 500 may proceed to termination as shown in step550, where operation of the linear drive system is disabled or otherwiseterminated.

Thus, it will be appreciated that a method or system disclosed hereinmay include operation in at least three distinct modes to accommodateintuitive user operation during initiation of a scan, controlled-forcescanning, and controlled-velocity exit from a scanning mode. Variationsto each mode will be readily envisioned by one of ordinary skill in theart and will fall within the scope of this disclosure. Thus, for exampleany one of the modes may be entered or exited by explicit user input. Inaddition, the method may accommodate various modes of operation usingthe sensors and other hardware described herein. For example thecontrolled-force mode 530 may provide for user selection or input of atarget force for controlled operation using, e.g., any of the userinputs described herein.

More generally, the steps described herein may be modified, reordered,or supplemented in a variety of ways. By way of example, thecontrolled-force mode of operation may include a controlled-velocitycomponent that limits a rate of change in position of the linear drivesystem. Similarly, the controlled-velocity mode for scan termination mayinclude a controlled-force component that checks for possible recoveryof controlled-force operation while returning the linear drive system toa neutral position. All such variations, and any other variations thatwould be apparent to one of ordinary skill in the art, are intended tofall within the scope of this disclosure.

In general, the systems described herein facilitate ultrasound scanningwith a controlled and repeatable contact force. The system may alsoprovide a real time measurement of the applied force when eachultrasound image is captured, thus permitting a variety of quantitativeanalysis and processing steps that can normalize images, estimate tissueelasticity, provide feedback to recover a previous scan state, and thelike. Some of these techniques are now described in greater detail.

FIG. 6 shows a process 600 for ultrasound image processing.

As shown in step 602, the process may begin with capturing a pluralityof ultrasound images of an object such as human tissue. In general, eachultrasound image may contain radio frequency echo data from the object,and may be accompanied by a contact force measured between an ultrasoundtransducer used to obtain the plurality of ultrasound images and asurface of the object. The contact force may be obtained using, e.g.,any of the hand-held, controlled force ultrasound scanners describedherein or any other device capable of capturing a contact force duringan ultrasound scan. The contact force may be manually applied, or may bedynamically controlled to remain substantially at a predetermined value.It will be appreciated that the radio frequency echo data may be, forexample, A-mode or B-mode ultrasound data, or any other type of dataavailable from an ultrasound probe and suitable for imaging. Moregenerally, the techniques described herein may be combined with anyforce-dependent imaging technique (and/or contact-force-dependentimaging subject) to facilitate quantitative analysis of resulting data.

As shown in step 604, the process 600 may include estimating adisplacement of one or more features between two or more of theultrasound images to provide a displacement estimation. A variety oftechniques are available for estimating pixel displacements intwo-dimensional ultrasound images, such as B-mode block-matching,phase-based estimation, RF speckle tracking, incompressibility-basedanalysis, and optical flow. In one aspect, two-dimensional displacementestimation may be based on an iterative one-dimensional displacementestimation scheme, with lateral displacement estimation performed atlocations found in a corresponding axial estimation. As described, forexample, in U.S. Provisional Application No. 61/429,308 filed on Jan. 3,2011 and incorporated herein by reference in its entirety,coarse-to-fine template-matching may be performed axially, withnormalized correlation coefficients used as a similarity measuresubsample estimation accuracy may be achieved with curve fitting.Regardless of how estimated, this step generally results in atwo-dimensional characterization (e.g., at a feature or pixel level) ofhow an image deforms from measurement to measurement.

It will be understood that feature tracking for purposes of displacementestimation may be usefully performed on a variety of differentrepresentations of ultrasound data. Brightness mode (or “B-mode”)ultrasound images provide a useful visual representation of a transverseplane of imaged tissue, and may be used to provide the features forwhich displacement in response to a known contact force is tracked.Similarly, an elastography images (such as stiffness or strain images)characterize such changes well, and may provide two-dimensional imagesfor feature tracking.

As shown in step 606, the process 600 may include estimating an inducedstrain field from the displacement. In general, hyperelastic models formechanical behavior work well with subject matter such as human tissuethat exhibits significant nonlinear compression. A variety of suchmodels are known for characterizing induced strain fields. One suchmodel that has been usefully employed with tissue phantoms is asecond-order polynomial model described by the strain energy function:

$\begin{matrix}{U = {{\sum\limits_{{i + j} = 1}^{2}\; {{C_{ij}\left( {I_{1} - 3} \right)}^{i}\left( {I_{2} - 3} \right)^{j}}} + {\sum\limits_{i = 1}^{2}\; {\frac{1}{D_{i}}\left( {J_{el} - 1} \right)^{2\; i}}}}} & \left\lbrack {{Eq}.\mspace{14mu} 1} \right\rbrack\end{matrix}$

where U is the strain energy per unit volume, I₁ and I₂ are the firstand second deviatoric strain invariant, respectively, and J_(el) is theelastic volume strain. The variables C_(ij) are the material parameterswith the units of force per unit area, and the variables D_(i) arecompressibility coefficients that are set to zero for incompressiblematerials. Other models are known in the art, and may be usefullyadapted to estimation of a strain field for target tissue ascontemplated herein.

As shown in step 608, the process 600 may include creating a trajectoryfield that characterizes a displacement of the one or more featuresaccording to variations in the contact force. This may includecharacterizing the relationship between displacement and contact forcefor the observed data using least-square curve fitting with polynomialcurves of the form:

x _(i,j)(f)=Σ_(k=0) ^(N)α_(i,j,k) ·f ^(k)  [Eq. 2]

y _(i,j)(f)=Σ_(k=0) ^(N)β_(i,j,k) ·f ^(k)  [Eq. 3]

where x_(i,j) and y_(i,j) are the lateral and axial coordinates,respectively of a pixel located at the position (i,j) of a referenceimage, and α and β are the parameter sets determined in a curve fittingprocedure. The contact force is f, and N denotes the order of thepolynomial curves. Other error-minimization techniques and the like areknown for characterizing such relationships, many of which may besuitably adapted to the creation of a trajectory field as contemplatedherein.

With a trajectory field established for a subject, a variety of usefulreal-time or post-processing steps may be performed, including withoutlimitation image correction or normalization, analysis of tissue changesover time, registration to data from other imaging modalities, feedbackand guidance to an operator/technician (e.g., to help obtain a standardimage), and three-dimensional image reconstruction. Without limiting therange of post-processing techniques that might be usefully employed,several examples are now discussed in greater detail.

As shown in step 610, post-processing may include extrapolating thetrajectory field to estimate a location of the one or more features at apredetermined contact force, such as to obtain a corrected image. Thepredetermined contact force may, for example, be an absence of appliedforce (i.e., zero Newtons), or some standardized force selected fornormalization of multiple images (e.g., one Newton), or any othercontact force for which a corrected image is desired, either forcomparison to other images or examination of deformation behavior. Withthe relationship between contact force and displacement provided fromstep 608, location-by-location (e.g., feature-by-feature orpixel-by-pixel) displacement may be determined for an arbitrary contactforce using Eqs. 2 and 3 above, although it will be appreciated that theuseful range for accurate predictions may be affected by the range ofcontact forces under which actual observations were made.

As shown in step 612, post-processing may include registering anundistorted image to an image of an object obtained using a differentimaging modality. Thus, ultrasound results may be registered to imagesfrom, e.g., x-ray imaging, x-ray computed tomography, magnetic resonanceimaging (“MRI”), optical coherence tomography, positron emissiontomography, and the like. In this manner, elastography data thatcharacterizes compressibility of tissue may be registered to othermedical information such as images of bone and other tissue structures.

As shown in step 614, post-processing may include comparing anundistorted image to a previous undistorted image of an object. This maybe useful, for example, to identify changes in tissue shape, size,elasticity, and composition over a period of time between imagecaptures. By normalizing a contact force or otherwise generatingcorrected or undistorted images, a direct comparison can be made fromone undistorted image to another undistorted image captured weeks,months, or years later.

As shown in step 616, post-processing may also or instead includecapturing multiple undistorted images of a number of transverse planesof an object such as human tissue. Where these images are normalized toa common contact force, they may be registered or otherwise combinedwith one another to obtain a three-dimensional image of the object. Theresulting three-dimensional image(s) may be further processed, eithermanually or automatically (or some combination of these), for spatialanalysis such as measuring a volume of a specific tissue within theobject, or measuring a shape of the tissue.

Still more generally, any post-processing for improved imaging,diagnosis, or other analysis may be usefully performed based on thequantitative characterizations of elastography described herein. Forexample, an ultrasound image of an artery may be obtained, and bymeasuring an amount of compression in the artery in response to varyingcontact forces, blood pressure may be estimated. Similarly, bypermitting reliable comparisons of time-spaced data, betterdiagnosis/detection of cancerous tissue can be achieved. Any suchultrasound imaging applications that can be improved with normalizeddata can benefit from the inventive concepts disclosed herein.

FIG. 7 is a schematic view of an ultrasound scanning system 700. Thesystem 700 may be used to capture an acquisition state for a handheldultrasound probe 702, such as any of the devices described herein,including quantitative data about the conditions under which the scanwas obtained. The conditions may include, e.g., a contact force of thehandheld ultrasound probe 702, apose of the ultrasound probe 702 inrelation to a target 704, and/or any other data that might be useful ininterpreting or further processing image data. In one aspect, data onthe contact force and/or pose may be used to obtain a three dimensionalreconstructed volume of a target.

The system 700 generally includes the handheld ultrasound probe 702 tocapture one or more ultrasound images of a target 704 through a skinsurface in a scan, a fiducial marker 708 applied to the skin surface ofthe target 704, and a camera 710. While a single fiducial marker 708 isdepicted, it will be understood that any number of fiducial markers,which may have identical or different features, may be used.

The ultrasound probe 702 may include an ultrasound imaging system 712that includes the at least one transducer 706 and a memory 714. Thememory 714 may be provided to store ultrasound data from the ultrasoundtransducer 706 and/or sensor data acquired from any of the sensorsduring an ultrasound scan. The ultrasound probe 702 may also include aforce control system 716.

The force control system may include a force sensor 718, a linear drivesystem 720, an input 722, and a controller 724, as described herein. Theforce sensor 718 may include a pressure transducer or the likeconfigured to obtain a pressure applied by the handheld ultrasound probe702 to the skin surface. The linear drive system 720 may be mechanicallycoupled to the handheld ultrasound transducer 706. The linear drivesystem 720 may include a control input 722 or be electronically coupledto the control input 722. The linear drive system 720 may be responsiveto a control signal received at the control input 722 to translate theultrasound transducer 706 along an actuation axis 114 as shown inFIG. 1. The linear drive system 720 may also include a position sensor744 that may provide positioning signals to the controller 724. Thecontroller 724 may be electronically coupled to the force sensor 718 anda control input of the linear drive system 720. The controller 724 mayinclude processing circuitry configured to generate the control signalto the control input 722 in a manner that maintains a substantiallyconstant predetermined contact force between the ultrasound transducer706 and the target 704, or a contact force that varies in apredetermined manner, all as discussed herein.

The ultrasound probe 702 may also include a sensor system 726 configuredto obtain a pose of the ultrasound probe 702. The sensor system 726 mayinclude the camera 710 and a lighting source 728, e.g., to capture imageof the fiducial marker 708 and or other visible features to obtaincamera motion data. In another aspect, the sensor system 720 may includeother sensors 730 such as one or more inertial sensors, range findingsensors (such as sonic, ultrasonic, or infrared range findingsubsystems), or any other circuitry or combination of circuitry suitablefor tracking relative positions of the ultrasound probe 702 and/or thetarget 704.

The system 700 may also include a processor 732 in communication withthe handheld ultrasound probe 702 and/or sub-systems thereof, eitherindividually or through a common interface 736 such as a wired orwireless interface for the ultrasound probe 702. It will be appreciatedthat a wide range of architectures are possible for control and dataacquisition for the system 700, including for example, a processor onthe ultrasound probe, a processor remote from the ultrasound probecoupled directly to one or more subsystems of the ultrasound probe, andvarious combinations of these. As such, the logical depiction of systemsin FIG. 7 should be understood as illustrative only, and any arrangementof components and/or allocation of processing suitable for an ultrasoundimaging system as contemplated herein may be used without departing fromthe scope of this disclosure. By way of non-limiting example, theprocessor 732 on the ultrasound probe 702 may be a controller or thelike that provides a programming interface for the force control system716, ultrasound imaging system 712, and/or sensor system 726, withsystem control provided by a remote processor (such as the process 742)through the interface 736. In another aspect, the processor 732 on theultrasound probe 702 may be a microprocessor programmed to control allaspects of the ultrasound probe 702 directly, with the remote processor742 providing only supervisory control such as initiating a scan ormanaging/displaying received scan data.

The processor 732 may be programmed to identify one or morepredetermined features (such as the fiducial marker 708 and/or otherfeatures on the skin surface of the target 704) and calculate a pose ofthe handheld ultrasound probe 702 using the one or more predeterminedfeatures in an image from the camera 710. In this manner, a number ofcamera images, each associated with one of a number of ultrasoundimages, may be used to align the number of ultrasound images in a worldcoordinate system. The ultrasound images, when so aligned, may becombined to obtain a reconstructed volume of the target 704.

The system 700 may also include one or more external sensor systems 734.The external sensor systems 734 may be integral with or separate fromthe sensor system 726. The external sensor system 734 may include anexternal electromechanical system coupled to the handheld ultrasonicprobe for tracking a pose of the handheld ultrasonic probe 702 throughdirect measurements, as an alternative to or in addition to image basedcamera motion data. The external sensor system 734 may also or insteadinclude an external optical system, or any other sensors used fortracking a pose of the ultrasound probe 702.

The system 700 also may include an interface 736, such as a wirelessinterface, for coupling the handheld ultrasound probe 702 in acommunicating relationship with a memory 738, a display 740, and aprocessor 742. The memory 738, the display 740, and the processor 742may be separate components or they may be integrated into a singledevice such as a computer. Where a wireless interface is used, varioustechniques may be employed to provide a data/control channel consistentwith medical security/privacy constraints, and/or to reduce or eliminateinterference with and/or from other wireless devices.

The display 740 may include one or more displays or monitors fordisplaying one or more ultrasound images obtained from the ultrasoundprobe 702 and/or one or more digital images recorded by the camera 710,along with any other data related to a current or previous scan.

The memory 738 may be used to store, e.g., data from the handheldultrasound probe 702 and the camera 710. The memory 738 may also storedata from other devices of the system 700 such as the sensors. Thememory 738 may store an acquisition state for an ultrasound image. Theacquisition state may for example include the pose and the pressure ofthe ultrasound transducer 706 during scanning, or data for recoveringany of the foregoing. The memory 738 may also or instead include a fixedor removable mass storage device for archiving scans and accompanyingdata. The memory 738 may be a remote memory storage device or the memorymay be associated with a computer containing the processor 742. Theinterface 736 may include a wireless interface coupling the handheldultrasound probe 702 in a communicating relationship with the remotestorage device, processor, and/or display.

The system 700 may include any other hardware or software useful for thevarious functions described herein. Also, the system 700 may be used forother applications including, for example, pathology tracking,elastography, data archiving or retrieval, imaging instructions, userguides, and the like.

FIG. 8 is a flowchart for a process 800 for obtaining a reconstructedvolume of a target using a handheld ultrasound probe.

As shown in step 802, a camera and ultrasound probe may be provided. Thecamera, which may be any of the cameras described herein, may bemechanically coupled in a fixed relationship to the handheld ultrasoundprobe in an orientation such that the camera is positioned to capture adigital image of a skin surface of a target when the handheld ultrasoundprobe is placed for use against the skin surface.

A lighting source may also be mechanically coupled to the ultrasoundprobe and positioned to illuminate the skin surface, and moreparticularly an area of the skin surface where the digital image iscaptured, when the handheld ultrasound probe is placed for use againstthe skin surface.

As shown in step 804, a fiducial marker with predetermined features suchas predetermined dimensions may be applied to the skin surface of atarget. The fiducial marker may be placed in any suitable location. Inone embodiment, the fiducial marker is preferably positioned at or nearthe location where the ultrasound probe will contact the surface so thatthe camera has a clear view of the fiducial marker.

As shown in step 806, an ultrasound image of the skin surface may beobtained from the handheld ultrasound probe. This may include data oncontact force, or any other data available from the ultrasound systemand useful in subsequent processing.

As shown in step 808, the camera may capture a digital image of thetarget, or the skin surface of the target. The digital image may includefeatures of the skin surface and/or the fiducial marker where thefiducial marker is within a field of view of the camera. The steps ofobtaining an ultrasound image from the handheld ultrasound probe andobtaining a digital image from the camera may be usefully performedsubstantially simultaneously so that the digital image is temporallycorrelated to the ultrasound image. In general, the ultrasound probe maycapture one or more ultrasound images and the camera may capture one ormore digital images in order to provide a sequence of images forming ascan. At least one of the digital images may include the fiducialmarker.

Although not depicted, a variety of supporting steps may also or insteadbe performed as generally described herein. The method may includewirelessly transmitting ultrasound data from the handheld ultrasoundprobe to a remote storage facility. The method may include displaying atleast one ultrasound image obtained from the handheld ultrasound probeand/or displaying at least one digital image recorded by the camera. Theimages may be displayed on one or more monitors, and may be displayedduring a scan and/or after a scan. In one aspect, where a sequence ofimages is obtained, a time stamp or other sequential and/orchronological indicator may be associated with each image (or imagepair, including the digital image from the camera and the ultrasoundimage from the probe). In this manner, a sequence of images may bereplayed or otherwise processed in a manner dependent onsequencing/timing of individual images.

As shown in step 809, a camera pose may be determined. By way of exampleand not limitation, this may be accomplished using motion estimation,which may be further based on a fiducial marker placed upon the skinsurface of a target. While the emphasis in the following description ison motion estimation using a fiducial, it will be understood thatnumerous techniques may be employed to estimate or measure a camerapose, and any such techniques may be adapted to use with the systems andmethods contemplated herein provided they can recover motion withsuitable speed and accuracy for the further processing described.Several examples are noted herein, and suitable techniques may include,e.g., mechanical instrumentation of the ultrasound probe, or image-basedor other external tracking of the probe.

As shown in step 810, a digital image may be analyzed to detect apresence of a fiducial marker. Where the fiducial marker is detected,the process 800 may proceed to step 814 where motion estimation isperformed and the camera pose recovered using the fiducial marker. Asshown in step 812, where no fiducial marker is detected, motionestimation may be performed using any other visible features of the skinsurface captured by the camera. However, determined, the camera pose maybe determined and stored along with other data relating to a scan. Itwill be understood that the “camera pose” referred to herein may be aposition and orientation of the actual digital camera, or any other poserelated thereto, such as any point within, on the exterior of, orexternal to the ultrasound probe, provided the point can be consistentlyrelated (e.g., by translation and/or rotation) to the visual imagescaptured by the digital camera.

Once a world coordinate system is established (which may be arbitrarilyselected or related to specific elements of the ultrasound system and/orthe target), a three-dimensional motion of the handheld ultrasound probewith respect to the skin surface for the digital image may be estimatedand the pose may be expressed within the world coordinates. Worldcoordinates of points on a plane, X=[X Y l] T, and the correspondingimage coordinates, x=[x y l]T may be related by a homography matrix.

This relationship may be expressed as:

[xyl]T=K[r ₁ r ₂ r ₃ [t][XYZl]T  [Eq. 4]

where K is the 3×3 projection matrix of the camera that incorporates theintrinsic parameters of the camera. The projection matrix may beobtained through intrinsic calibration of the camera. The rotationmatrix R and the translation vector t describe the geometricrelationship between the world coordinate system and the imagecoordinate system, and r₁, r₂, and r₃ are the column vectors of R.

World coordinates of points on a planar structure may be related by a3×3 planar homography matrix. For points on a planar structure, Z=0 andthe relationship may be expressed as:

[xyl]T=K[r ₁ r ₂ [t][XYZl]T  [Eq. 5]

The image coordinates in different perspectives of a planar structuremay also be related by a 3×3 planar homography matrix. An image-to-worldhomography matrix may then be used to show the relationship between thecamera images (the image coordinate system) and the ultrasound images (aworld coordinate system). The homography matrix may be expressed by theformula x′=Hx, where x′=[x′y′l′] and x=[x y l], points x′ and x arecorresponding points in the two coordinate systems, and His thehomography matrix that maps point x to x′. H has eight degrees offreedom in this homogeneous representation and H can be determined by atleast four corresponding points. H may be written as a 9-vector matrix,[h₁₁, h₁₂, h₁₃, h₂₁, h₂₃, h₃₁, h₃₂, h₃₃]T, with n corresponding points,x_(i)′ and x_(i) for i=1, 2, 3, . . . n. This matrix may be expressed bythe formula A_(h)=O, where A may be a 2n×9 matrix:

$\begin{matrix}{A = \begin{matrix}x_{1} & y_{1} & 1 & 0 & 0 & 0 & {{- x_{1}}x_{1}^{\prime}} & {{- y_{1}}x_{1}^{\prime}} & {- x_{1}^{\prime}} \\0 & 0 & 0 & x_{1} & y_{1} & 1 & {{- x_{1}}y_{1}^{\prime}} & {{- y_{1}}y_{1}^{\prime}} & {- y_{1}^{\prime}} \\\vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots \\x_{n} & y_{n} & 1 & 0 & 0 & 0 & {{- x_{n}}x_{n}^{\prime}} & {{- y_{n}}x_{n}^{\prime}} & {- x_{n}^{\prime}} \\0 & 0 & 0 & x_{n} & y_{n} & 1 & {{- x_{n}}y_{n}^{\prime}} & {{- y_{n}}y_{n}^{\prime}} & {- y_{n}^{\prime}}\end{matrix}} & \left\lbrack {{Eq}.\mspace{14mu} 6} \right\rbrack\end{matrix}$

The solution his the unit eigenvector of the matrix ATA with the minimumeigenvalue. The homography matrix may be estimated by any suitablemethod including, for example, using a RANSAC (Random Sample Consensus)algorithm.

If the image to world homography matrix and the projection matrix of thecamera are known, then the camera pose in the world coordinate systemmay be calculated for each image i, where i=1, 2, 3, . . . n. Thus,column vectors r₁, r₂, and t can be calculated. Vector r₃ may beexpressed as the formula r₃=r₁×r₂ since R is a rotation matrix.

After the points or other features on the fiducial marker are selected,the corresponding features may be identified for each digital imagewhere the fiducial marker is present. In general, the fiducial markermay be visible in at least one of the plurality of digital images (andnot necessarily the first image), although it is also possible torecover the camera pose entirely based on features other than thefiducial marker. The planar homography matrix from image i to theprevious image may be calculated. The correspondences betweenconsecutive images may be extracted using any suitable technique such asscale-invariant feature transform (SIFT).

As shown in step 816, the ultrasound image(s) may be aligned in theworld coordinate system. The process 800 may also include a step ofcalibrating the ultrasound probe so that ultrasound images can beconverted to camera coordinates and/or the world coordinate system, thuspermitting the ultrasound image(s) to be registered in a commoncoordinate system. For each ultrasound image, image coordinates may beconverted to world coordinates using the corresponding estimates ofcamera pose. Transformations derived from ultrasound calibration mayalso be used to convert the image coordinates to the world coordinates.A plurality of ultrasound images may be aligned to the world coordinatesystem based on the corresponding camera poses. The resulting ultrasoundimages may be further processed in three dimensions, for example toobtain a shape and/or volume of the target, or features and/or objectswithin the target from the plurality of ultrasound images that have beenaligned in the world coordinate system. Thus, for example, a shape orvolume of tissue, a tumor, or any other object within the target may becalculated in three-dimensions based on the registered ultrasoundimages, particularly where each image is normalized as described hereinto a single contact force.

A processor in communication with the ultrasound probe and the cameramay be configured to perform the steps described herein including:identifying the fiducial marker in two or more digital images from thecamera; establishing a world coordinate system using the predetermineddimensions of the fiducial marker and the two or more digital images;estimating a three dimensional pose of the handheld ultrasound probewith respect to the two or more images; and aligning the plurality ofultrasound images in the world coordinate system to obtain areconstructed volume of the target. It will be readily appreciated thatthe steps may be performed any number of times for any number of cameraimages and/or ultrasound images, and processing may be performed as newimages are acquired, or in a post processing step according to thecapabilities of the system or the preferences of a user.

FIG. 9 is a flowchart of a process for capturing an acquisition statefor an ultrasound scan.

As shown in step 902, the process 900 may begin with capturing anultrasound image with a handheld ultrasound probe such as any of theprobes described herein. The steps of the process 900 may in generalperformed a single time, or may be repeated any number of timesaccording to the intended use(s) of the acquisition state data.

As depicted generally in step 903, the process 900 may include capturingacquisition state data. While various specific types of acquisitionstate data are described herein, it will be appreciated that moregenerally any data related to the state of an ultrasound probe, thetarget, the surrounding environment, and the like may be usefullyacquired as acquisition state data and correlated to one or moreultrasound images. Thus, for example, the acquisition state may includelocation, orientation, orientation, velocity (in any of the foregoing),acceleration, or any other intrinsic or extrinsic characteristics of thecamera, the ultrasound probe, the target, or combinations of theforegoing. Similarly, an acquisition state may include environmentalfactors such as temperature, humidity, air pressure, or the like, aswell as operating states of hardware, optics, and the like. Still moregenerally, anything that can be monitored or sensed and usefullyemployed in an ultrasound imaging system as contemplated herein may beused as an acquisition state.

As shown in step 904, capturing acquisition state data may includecapturing a pose of the handheld ultrasound probe, which may beperformed substantially concurrently with the step of obtaining theultrasound image, and may include any of the techniques describedherein. This step may also or instead employ other techniques formeasuring position and orientation of an ultrasound probe with respectto a skin surface. For example, the present process may be combined withtechniques involving the use of accelerometers or gyroscopes, ortechniques involving the level of speckle dissimilarity or decorrelationbetween consecutive scans.

As shown in step 906, capturing acquisition state data may also orinstead include capturing a contact force of the handheld ultrasonicprobe, which may be performed substantially concurrently with the stepof obtaining the ultrasound image.

As shown in step 908, the process 900 may include normalizing theultrasound images to a common contact force. As shown in step 910, theprocess 900 may include storing ultrasound image data and anyacquisition state data (e.g., pose and contact force) temporally orotherwise associated therewith. As shown in step 912, the process 900may include displaying one or more ultrasound images and/or acquisitionstate data or the like. As shown in step 914, the process 900 mayinclude aligning the ultrasound image(s) in a world coordinate system,such as by using the acquisition state data. As shown in step 916, theprocess 900 may include reconstructing a volume of the target, or anobject or feature within the target, using a plurality of ultrasoundimages. This may be accompanied by a display of the reconstructedvolume. More generally, any use of the various types of acquisitionstate data described herein for control, image enhancements, or othervisualizations and/or data processing may be incorporated into theprocess 900.

FIG. 10 shows a fiducial marker that may be used with the techniquesdescribed herein. The fiducial marker 1000 may be applied to a skinsurface of a target as a sticker, or using ink or any other suitablemarking technique. In one aspect, the fiducial marker 1000 may be black,or any other color or combination of colors easily detectable usingdigital imaging techniques. In another aspect, the fiducial marker 1000may be formed of a material or combination of materials transparent toultrasound so as not to interfere with target imaging. The fiducialmarker 1000 may also have a variety of shapes and/or sizes. In oneembodiment, the fiducial marker 1000 may include an inner square 1002 ofabout three millimeters in height, and one or more outer squares 1004 ofabout one millimeter in height on the corners of the inner square 1002.In this manner, several corner features 1006 are created for detectionusing an imaging device.

While the systems and methods above describe specific embodiments ofacquisition states and uses of same, it will be understood thatacquisition state data may more generally be incorporated into anultrasound imaging workflow, such as to provide operator feedback orenhance images. Thus, in one aspect there is disclosed herein techniquesfor capturing an acquisition state for an ultrasound scan and using theacquisition state to either control an ultrasound probe (such as withforce feedback as described herein) or to provide user guidance (such asto direct a user through displayed instructions or tactile feedback to aprevious acquisition state for a patient or other target). The improvedworkflows possible with an ultrasound probe that captures acquisitionstate are generally illustrated in the following figure.

FIG. 11 shows a generalized workflow using acquisition states. Ingeneral, an ultrasound probe such as any of the devices described hereinmay capture image data and acquisition state data during an ultrasoundscan. In one aspect, this data may be fed directly to a user such as anultrasound technician during a scan. For example, ultrasound imagesand/or acquisition state data may be displayed on a display of acomputer or the like while a scan is being performed.

In another aspect, machine intelligence may be applied in a variety ofmanners to augment a scanning process. For example, acquisition statedata concerning, e.g., a pose of the ultrasound probe may be used tocreate a graphical representation of a scanner relative to a target, andthe graphical representation may be depicted on the display showing arelative position of the ultrasound probe to the target in order toprovide visual feedback to the user concerning orientation. As anotherexample, contact force may be displayed as a numerical value, and/or anultrasound image with a normalized contact force may be rendered forviewing by the user during the scan. As another example, the contactforce may be provided as a visual or audio indication on the probe asfurther described in FIG. 16. In another aspect, a desired acquisitionstate may be determined (e.g., provided by a user to the computer), andthe machine intelligence may create instructions for the user that canbe displayed during the scan to steer the user toward the desiredacquisition state. This may be a state of diagnostic significance,and/or a previous acquisition state from a current or historical scan.In another aspect, the desired acquisition state may be transmitted ascontrol signals to the ultrasound probe. For example, control signalsfor an instantaneous contact force may be communicated to aforce-controlled ultrasound device such as the device described above.This may also or instead include scanning data parameters such asfrequency or array beam formation, steering, focusing, and the like.

In addition, the capability of capturing a multi-factor acquisitionstate including, e.g., contact force and position permits enhancementsto analysis and diagnostic use of an ultrasound system. For example,analysis may include elastography, image normalization,three-dimensional reconstruction (e.g., using normalized images), volumeand/or shape analysis, and the like. Similarly, diagnostics may beimproved, or new diagnostics created, based upon the resulting improvedultrasound images as well as normalization of images and accurateassessment of an acquisition state. All such uses of an ultrasoundsystem having acquisition state capabilities, feedback controlcapabilities, and machine intelligence as contemplated herein and areintended to fall within the scope of this disclosure.

FIG. 12 depicts the use of hard limits 1202 and soft limits 1208 incontrolling the travel limits of a linear drive system or similar drivesystem for the probe 1210. In general, a system for controlling theprobe 1210 may selectively switch between a force-control algorithm anda position-control algorithm to enforce a position of the linear drivesystem away from the hard limits and/or within a desired operatingrange.

Hard limits 1202 may be defined by limit switch(es) 1204 as previouslydescribed. The limit switch(es) 1204 may define physical limits oftravel of the linear drive system 122. The limit switch(es) 1204 may beelectronically coupled to the controller 120 and provide a signal to thecontroller 120 to indicate when the limit switch(es) 1204 detects an endof travel of the linear drive system 122 along the actuation axis 114.This may be used, for example, to immediately stop motion by the lineardrive system 122 when one of the limit switches 1204 is contactedregardless of other control information. It will be understood that alimit switch 1204 is preferably positioned before a true, physical hardlimit of travel such as a mechanical barrier to further movement inorder to prevent travel to and potentially beyond a position that mightdamage the linear drive system 122 or otherwise cause a malfunction.Thus, while the following discussion may describe the limit switches1204 as representing or defining the hard limits 1202 themselves, theymay also or instead include any proxy for the physical hard limits forthe device 1200 that permits control of the linear drive system 122relative to the hard limits 1202. Any such meaning consistent with suchuse of the limit switches 1204 is intended to fall within the scope ofthis description.

The soft limits 1208 may include software defined positions of thelinear drive system 122 along an actuation axis 114, which may be apredetermined distance from the hard limit 1202 of the limit switch (es)1204. The soft limits 1208 may be enabled or disabled, and may have alocation set at any predetermined position(s) along the operating rangeof the linear drive system 122. In general, the soft limits 1208 may bea predetermined system parameter that may be stored in a memory of theprobe. As previously described, the linear drive system 122 may includea position sensor 142 that may be electronically coupled to a positioncontroller. Additionally, as previously described, the probe 1210 mayinclude a force/torque sensor 110 to measure the force applied to thetarget 136. In embodiments, the force sensor 110 may provide signals toa force control module and the position sensor 142 may provide signalsto a position control module. In embodiments, the force control moduleand position control module may be implemented in separate controllers,in a common controller, part of the probe controller 120, or any othersuitable controller configuration. The linear drive system 122 may useboth the force control module signal and the position control modulesignal, either separately or in combination, to calculate controlsignals to be sent to the linear drive system 112 for positioning of theprobe 1210.

During operation of the probe 1210 within the normal operation range1212 of the linear drive system 122, the controller may receive forceand position signals from the force sensor 110 and position sensor 142respectively. While in the normal operation range 1212, the controller120 may monitor the position of the probe 1210 relative to the softlimits 1208 while a force control loop actively maintains force with acontrol signal to the linear drive system 122. During normal use, theultrasound technician may apply a force to the probe 1210 that isgreater or less than a predetermined desired force. The force controlmodule may receive signals from the force sensor 110 and the forcecontrol module may provide a control signal to indicate a move of theprobe along the linear drive system 122 to maintain the predetermineddesired force on the target 136. For example, the force control modulemay provide control signals to move the probe 1212 closer to the target136 to increase the force applied to the target 136 or may signal a moveaway from the target 136 to decrease the force applied to the target.During operation of the probe 1210, the ultrasound technician mayincrease or decrease the applied force in a manner that causes the probe1210 to move along the linear drive system 122 to a position near one ofthe soft limits 1208.

As the probe 1210 approaches one of the soft limits 1208, the controller120 may activate both the force control module and position controlmodule to provide a combined signal to control the positioning of theprobe 1210 within the linear drive system 122. Once at the soft limit1208, the controller 120 may sum the output signals from both the forcecontrol module and the position control module to generate the controlsignal output to the linear drive system 122. In general, this approachto control balances a goal of maintaining the predetermined contactforce with a goal of avoiding the hard limit 1202 of the limit switch1204.

In embodiments, once the probe 1210 has reached the soft limits 1208,the controller may balance the control signal output using an endpointavoidance strategy. As one example, the endpoint avoidance strategy mayallow the ultrasound technician to reach the soft limit 1208, but as thetechnician maintains the position at the soft limit 1208, the controllermay increase the predetermined force setting and maintain the positionof the probe 1210 at the soft limit 1208. In one aspect, as theultrasound technician moves away from the soft limit 1208, after anindication of reaching the soft limit, the predetermined force settingmay be maintained at the new increased force level. This controltechnique advantageously permits the ultrasound technician to change thepredetermined force setting (e.g., a target force) without having tochange the software configuration, press buttons, or use any othermethod of adjusting the predetermined force setting. The technician mayreceive an indication that the predetermined force setting has beenchanged to a new setting. The indication may be a sound, light,vibration, or other indication. The new predetermine force setting maybe indicated on the display 230.

As a second example of an endpoint avoidance strategy, once theultrasound technician reaches the soft limit 1208, the controller mayincrease the predetermined force setting and maintain the position ofthe probe 1210 at the soft limit 1208. However, as the ultrasoundtechnician moves away from the soft limit 1208, after an indication ofreaching the soft limit 1208, the controller may proportionally increaseor decrease the predetermined force setting back to the originalpredetermined force setting. As the technician moves farther away fromthe soft limit 1208 the original predetermined force setting may bereached and scanning may resume at the target contact force.

In embodiments, the endpoint avoidance strategies may be usedindividually, or in combination. For example, one endpoint avoidancestrategy may be used at the soft limit 1208 of the greatest appliedforce and a different endpoint avoidance strategy may be used at thesoft limit 1208 of the least applied force. The endpoint avoidancestrategies may be fixed or may be software definable within aconfiguration system. In some implementations, the endpoint avoidancestrategy includes ramping up a target force for the linear drive system,such as after engagement with the target surface.

It should be understood that these endpoint avoidance strategies areprovided as examples of strategies and other strategies may be usedwithout departing from the scope of this disclosure. In general, hardand soft limits may be used as control inputs in a variety of ways toprovide control over the linear drive system according to where, withina range of motion, the system is currently operating.

FIG. 13 depicts a graph showing control signals varying according toprobe position for three different operation modes. For the operation ofthese modes a Proportional-Derivative (PD) controller may be used togenerate an output signal to control the linear drive system 122. Inmode 1 (1302), the position controller module provides position controlthat becomes active to avoid endpoints when the linear drive system ofthe probe 1210 reaches the soft limits 1208. As shown in graphicaldepiction of mode 1 (1302), the position output signal is linearlyproportional to the probe 1210 position once the soft limits are reachedand the position controller module is enabled. As shown in mode 2(1304), the force controller module is constantly enabled and results ina linear response to contact force centered about the center of travel.In mode 3 (1308), the responses of the position controller module andthe force controller are combined (either linearly combined ornon-linearly). By using a non-linear combination, the transition may besmoothly affected at a location between the center and the softlimit(s). It should be understood that the transition between, orcombination of, operating modes may be realized in a variety of ways toobtain a comfortable user experience without departing from the scope ofthis disclosure.

FIG. 14 depicts a block diagram of a combined force and position controlmodule 1400. In embodiments, the force control module and positioncontrol module may be implemented as separate control modules or as acombined module as depicted in FIG. 14. In an embodiment where the forceand position modules are implemented separately, a summing circuit 1410may be implemented as a separate module to receive signals from theforce and position modules and then output a resulting summed contactforce 1408. The summing circuit 1410 may optionally weight inputs in anysuitable way to effect a smooth transition between the two potentiallyconflicting control loops for force and position. The primary input maybe a predefined force setting F_(target) 1402 for the contact force. Inaddition, a predefined position target, X_(target) 1404, may be used. Inthis embodiment, the F_(target) and X_(target) may each be provided asinputs to a PD controller and summed in a summing circuit 1410 using anysuitable weighting scheme as appropriate. The resulting summed contactforce 1408 may be output to the linear drive system 122 to position theprobe 1210. As previously described, the force control module andposition control module may be enabled or disabled individually basedon, e.g., a current position or a current contact force, depending onthe endpoint avoidance strategy selected to control the probe 1210. Inembodiments, the user may be able to manually select the endpointavoidance strategy or strategies.

FIG. 15 depicts a system to provide an indication of the probe positionrelative to the end of travel. As the technician uses the probe 1210 andapproaches one of the endpoints, visual and/or audio indications ofposition may be provided to the technician. By providing theseindications, the technician may be able avoid endpoints, or avoid thetriggering of the endpoint avoidance strategies, so that a consistentcontact force is applied to the target 136. A probe position indicatormay be used so that the technician can determine a position of the probe1210 relative to the center of the linear drive system (or otherlocation) without having to look away from the probe 1210 or the target136.

One or more LEDs 1502, or a similar sort of light or visible indicator,may be provided that represent the available travel distance of theprobe 1210. One LED 1504 may be left dark as an indicator of theposition of the probe 1210 within the available travel distance, whilethe remaining LEDs are illuminated. Other lighting schemes may also orinstead be used, such as by illuminating a single one of the LEDs 1502to indicate position, or by having the one LED 1504 illuminate with adifferent color (e.g., where all LEDs 1502 are green, except for the oneLED 1504 which is red). As the technician applies a force that isdifferent from the predetermined force setting, the position of thesingle LED (i.e., the LED of a different colored, or illuminated, orleft dark) may move along the column of LEDs 1502 to provide a visualindication of the current status. In another aspect, as the probe 1210approaches an end of travel (e.g., soft or hard limit) the correspondingLED may be displayed in a different color, red for example. The LEDcolumn may be color coded in this manner using a variety of visualtechniques. For example, the center region may be green, the areabetween the soft limits and hard limits (e.g., where position control isactive) may be yellow), and regions at or near the hard limits may bered. In this manner, the technician may have a continuous indication ofthe position of the probe 1210 relative to the useful operating range.

In an embodiment, the LEDs 1502 may provide a direct indication of theapplied force (e.g., relative to the target contact force) using any ofthe color schemes described above, with each LED representing a portionof a useful operating range for the probe 1210. The force range for aparticular LED may be a fixed value or may be set by the technician aspart of a configuration system, and may be a relative value (e.g.,according to the target contact force, the soft limits, etc.) or anabsolute force (e.g., a range of forces in Newtons or other units). Forexample, each LED 1502 may represent a force of 1 Newton. By way ofexample, if there are ten LEDs 1502 used in the set of LEDs then theprobe travel limits would be +/−5 Newtons.

Limits of the linear drive system, or more generally, operation of theprobe 1210, may also or instead be signaled using other techniques. Forexample, an audible sound may be provided as an indication of the probeapproaching an end of travel. The sound may be varied in intensity asthe technician approaches or moves away from one of the endpoints.Similarly a piezoelectric buzzer or other audio/visual signal may beemployed.

In certain ultrasound imaging applications, the contact angle is alsoimportant. Thus, when using an ultrasound probe, it may be useful toknow that the probe is being used within certain angles relative to thetarget 136, e.g., within 20 degrees of normal for example. An indicatorsuch as any of the audio/visual indicators described above may be usedto indicate when the probe has moved into or out of such target ranges.Where LEDs are used, the LEDS may be arranged to provide informationconcerning where, across a range of possible contact angles, the probeis currently operating, and the LEDs may be illuminated (as with contactforce above) to provide visual feedback concerning a current state ofthe probe. A target angle may be any predetermined angle, and mayinclude a fixed angle or a software-defined angle within a configurationsystem.

It should be understood that the various signaling techniques describedabove may be used alone or in any combination to improve user feedbackfor a force-controlled ultrasound probe.

FIG. 16 depicts the placement of pushbuttons 1602 on the probe. In oneembodiment, these pushbuttons may have a dedicated use such as to recorddata, zeroing the orientation angle, varying the predetermined forcesetting, turning the probe on and off, or other functions. In anotheraspect, the pushbuttons 1602 may be software definable in aconfiguration system so that the pushbuttons 1602 may provide differentfunctions depending on the operation being performed, or based upon userpreferences. The technician may be able to set the function for apushbutton 1602 prior to using the probe or during the use of the probe.

Other details and embodiments are described by way of example and not oflimitation in U.S. Provisional Application No. 61/715,406, the entiretyof which is hereby incorporated by reference.

It will be appreciated that many of the above systems, devices, methods,processes, and the like may be realized in hardware, software, or anycombination of these suitable for the data processing, datacommunications, and other functions described herein. This includesrealization in one or more microprocessors, microcontrollers, embeddedmicrocontrollers, programmable digital signal processors or otherprogrammable devices or processing circuitry, along with internal and/orexternal memory. This may also, or instead, include one or moreapplication specific integrated circuits, programmable gate arrays,programmable array logic components, or any other device or devices thatmay be configured to process electronic signals. It will further beappreciated that a realization of the processes or devices describedherein may include computer-executable code created using a structuredprogramming language such as C, an object oriented programming languagesuch as C++, or any other high-level or low-level programming language(including assembly languages, hardware description languages, anddatabase programming languages and technologies) that may be stored,compiled or interpreted to run on one of the above devices, as well asheterogeneous combinations of processors, processor architectures, orcombinations of different hardware and software. At the same time,processing may be distributed across devices such as the handheld probeand a remote desktop computer or storage device, or all of thefunctionality may be integrated into a dedicated, standalone deviceincluding without limitation a wireless, handheld ultrasound probe. Allsuch permutations and combinations are intended to fall within the scopeof the present disclosure.

In other embodiments, disclosed herein are computer program productscomprising computer-executable code or computer-usable code that, whenexecuting on one or more computing devices (such as the controllerdescribed above), performs any and/or all of the steps described herein.The code may be stored in a computer memory or other non-transitorycomputer readable medium, which may be a memory from which the programexecutes (such as internal or external random access memory associatedwith a processor), a storage device such as a disk drive, flash memoryor any other optical, electromagnetic, magnetic, infrared or otherdevice or combination of devices. In another aspect, any of theprocesses described herein may be embodied in any suitable transmissionor propagation medium carrying the computer-executable code describedherein and/or any inputs or outputs from same.

While particular embodiments of the present invention have been shownand described, it will be apparent to those skilled in the art thatvarious changes and modifications in form and details may be madetherein without departing from the spirit and scope of this disclosureand are intended to form a part of the invention as defined by thefollowing claims, which are to be interpreted in the broadest senseallowable by law.

What is claimed is:
 1. A system comprising: a handheld ultrasound probeincluding a grip and one or more ultrasound transducers to capture anultrasound image of a target through a skin surface; a force sensorconfigured to provide a force signal representative of a force appliedby the handheld ultrasound probe to the skin surface; a position sensorconfigured to provide a position signal representative of the handheldultrasound probe position within a linear drive system; a linear drivesystem configured to move the one or more ultrasound transducersrelative to the grip in response to a control signal; a hard limit formovement of the linear drive system identified by a hard limit switchthat detects when the linear drive system has physically reached thehard limit; a soft limit for movement of the linear drive system betweenthe hard limit and a range of positions available to the linear drivesystem, the soft limit stored in a memory; and a controller moduleconfigured to generate the control signal according to a force-controlalgorithm when the linear drive system is at a position further from thehard limit than the soft limit and to generate the control signal to thelinear drive system using an endpoint avoidance strategy when the lineardrive system is between the hard limit and the soft limit.
 2. The systemof claim 1 wherein the endpoint avoidance strategy includes summing theforce signal and position signal to generate a combined positioningcommand as the control signal to the linear drive system.
 3. The systemof claim 1 wherein the endpoint avoidance strategy includes selectivelyswitching between a force-control algorithm and a position-controlalgorithm to enforce a position of the linear drive system away from thehard limit.
 4. The system of claim 1 further comprising a second hardlimit and a second soft limit, wherein the soft limit and the secondsoft limit are at positions along a range of travel of the linear drivesystem between the hard limit and the second hard limit.
 5. The systemof claim 4 wherein the controller module employs a different endpointavoidance strategy for the second soft limit than the first soft limit.6. The system of claim 1 wherein the force-control algorithm maintains apredetermined force against the skin surface.
 7. The system of claim 6wherein the endpoint avoidance strategy includes increasing thepredetermined force for the force-control algorithm from an initialtarget value.
 8. The system of claim 7 wherein the predetermined forceis returned to the initial target value when the position of the lineardrive system returns to the soft limit.
 9. The system of claim 1 furthercomprising one or more LEDs programmed to provide an indication ofapplied force.
 10. The system of claim 1 further comprising one or moreLEDs configured to provide an indication of position within a range oftravel of the linear drive system.
 11. The system of claim 1 furthercomprising an alert transducer to signal when a soft limit has beencrossed.
 12. The system of claim 1 further comprising an alerttransducer to signal when the linear drive system is approaching a hardlimit.
 13. The system of claim 12 wherein the alert transducer includesone or more of an audible alert, an LED, and a buzzer to vibrate thehandheld ultrasound probe.
 14. The system of claim 1 wherein theendpoint avoidance strategy includes ramping up a target force for thelinear drive system.
 15. A system comprising: a handheld ultrasoundprobe including a grip and one or more ultrasound transducers to capturean ultrasound image of a target through a skin surface; a force sensorconfigured to provide a force signal representative of a force appliedby the handheld ultrasound probe to the skin surface; a position sensorconfigured to provide a position signal representative of the handheldultrasound probe position within a linear drive system; a linear drivesystem configured to move the one or more ultrasound transducersrelative to the grip in response to a control signal; and a controllermodule configured to execute a force-control algorithm to control anapplied force of the handheld ultrasound probe to the skin surface and aposition-control algorithm to control a position of the linear drivesystem, the controller further configured to selectively switch betweenthe force-control algorithm and the position-control algorithm accordingto one or more predetermined inputs.
 16. The system of claim 15 whereinthe linear drive system includes two endpoints, and wherein thecontroller module uses an endpoint avoidance strategy for the lineardrive system to selectively switch between the force-control algorithmand the position-control algorithm.
 17. The system of claim 16 whereinthe endpoint avoidance strategy includes summing the force signal andposition signal to generate a combined positioning command as thecontrol signal to the linear drive system.
 18. The system of claim 15wherein the force-control algorithm maintains a predetermined forceagainst the skin surface.
 19. The system of claim 15 further comprisingone or more LEDs programmed to provide an indication of applied force.20. The system of claim 15 further comprising one or more LEDsconfigured to provide an indication of position within a range of travelof the linear drive system.